For those interested in this continuing debate, Andrew Leigh and Justin Wolfers have a new paper (PDF) comparing the performance of polls and betting markets in predicting election outcomes.
For what it’s worth, I think the two are about equally good, at least when an election is about to happen. There’s no indication that markets have significant private information: for example, they react, and sometimes overreact to ‘news’ that turns out, in retrospect, to be misleading. But most of the time, they provide a pretty good summary of available public information.
This is not too surprising to me. Although I’m strongly of the view that financial markets are not fully efficient in the semi-strong sense of making optimal use of all public information, the violations are subtle (but important!). Tests of election markets simply don’t have the resolution to pick up subtle violations, as opposed to occasional single-point observations, for example, the collapse of the Bush bet when the first exit polls on election day suggested a Kerry win.
John Quiggin 
………………….
Pr Q keeps on harping on the single instance in recent history which markets failed to properly account for, the exit poll fiasco on election day. This is a very weak, even invalid criticism.
The exit polls are retrodictive tools which measure past behaviour. Electoral betting markets are predictive tools designed to guess future behaviour. So Pr Q is simply accusing electoral markets of misreporting a race-in-progress, which could have been fixed by a last minute “nobble”.
The fact that the Bush bet collapsed in the midst of exit polling, because early indicators prematurely suggested a Kerry surge, does not prove the disutility of electoral markets. It just shows that electoral markets are not omniscient and were not able to account for freak events, in this case the propensity of DEM voters to show up early and vote. Neither polls nor pundits gave any prior indication of this occurring.
Pr Q should compare apples with apples. If Pr Q had compared the electoral market trend line with the aggregated polling survey trends for the same prior period, say from the nomination of GW Bush until the establishment of the first homeland polling station, then he would have found the electoral markets were much better predictors.
Time and again electoral markets have outperformed polls & pundits, as they did in both the 2004 AUS and US elections. They are better predictors because when you are putting your money where your mouth is, you tend to choose more wisely.
If you read the Leigh-Wolfers paper, you’ll see that they don’t claim that markets outperformed polls in the 2004 Aus election. As in the previous election, the betting odds were pretty much equal to the proportion of polls predicting a Coaltion win.
I could not read the Leigh-Wolfers paper as the proffered link did not work (errant comma in the URL). Having now read it, the paper points out that the polls had Howard as a weak favourite whereas the pundits had him as a strong favourite throughout the campaign. Correct link:
Given Howard came home with an increased majority and control of the Senate I’d say that puts the punters ahead of the polls.
I would also hazard a guess that the punters trounced the pollsters over the course of the US campaign, freak events like the exit poll hoax notwithstanding.
I dont maintain that punters are omniscient, only that they are better than pollsters because quality of data processing is better value than quantity.
Jack, you are correct that Wolfers and Leigh do claim at some points that markets outperformed polls. But I repeat that their evidence doesn’t support this, and their formal conclusions are very weak. The fact that the polls didn’t predict an increased Howard majority is not very interesting in the absence of a market price on this outcome.
A real challenge would have been to predict the outcome of the WA election before the campaign began. The polls were certainly way off on this one, but I’m pretty sure the same was true of the markets.
Simon Jackman’s research also argued the opinion polls did a poor job in the lead up to the 2004 election, with three of the five polling organisations demonstrating a systemic pro-Labor bias.
John, another way to compare the two – which we didn’t explicitly push – is to look at volatility in the poll predictions. All five bookmakers thought Howard was going to win on every single day (going back to mid-2003). The polls mostly thought Labor would win, but were erratic throughout the cycle and even through the election campaign. So if you picked any random day from mid-2003 to polling day and compared bookies and pollsters, the bookie was right 100% of the time, and the pollster was generally wrong.
“The polls mostly thought Labor would win”
This is not correct. The polls ask the question: “who would you vote for if an election was held this Saturday?”. Since, except in the last week of an election campaign, an election is not going to be held the following Saturday, people are not being asked who they would vote for in an actual election. It’s one thing to say this or that to a pollster, even if people mean what they say at the time they say it. It’s another thing altogether to mark your ballot paper when the future of the country is at stake.
This is why mid term polls have so little predictive power. People are being asked a hypothetical question with few if any practical consequences, and they know it.
And even if polls really are a good predictor of voting intentions at an actual election, governments change their policies in response to the polls to make themselves more popular.
We saw this clearly in 2001. Early in 2001, the polls were terrible for Howard, and he almost certainly would have lost an election had one been held then. He responded to the polls by freezing petrol excise and spending big in the budget, and his poll numbers started to improve. Add a slice of luck (Tampa, S11) and he won easily. You can’t blame the polls held early in the year for not predicting the election result.
At most, you could say that the betting markets correctly anticipated that Howard would respond to his unpopularity by doing things that would make him popular again. But there’s no way the polls could have done that, not because they are inaccurate, but because they ask a different question.
I seem to recall that most sportsbetting market participants actually got the 2001 election wrong, but the weight of money correctly backed Howard. The Iowa markets also see a small number of “strong hands” dominate the market. What I think is remarkable about these markets is that they produce good predictive results despite rather than because of the “wisdom of crowds.” I hope Andrew will look into this issue in more detail!
……………….
I just can’t reconcile this statement with the evidence on public record. Punters generally supported incumbents whilst polls were divided.
The Morgan poll was famously off in the 2001 AUS election, predicting a Beazley victory in the days before that vote. A number of polls had Howard behind during the 2004 AUS election campaign.
It is true that most of the aggregated national polls had Bush barely ahead during the 2004 US election. But the disaggregaged state polls had Kerry ahead in electoral college votes.
The data from the UK is at the mo very interesting – 2 days ago ICM put labor well ahead but yesterday the betting markets took a lot of money on the Tories. Today Mori Puts the Tories neck and neck with Labor.
It’d be interesting to see the extent to which poll data is now valuable.
Punters base their investment choices on more or less refined algorythms of previous voter behaviour. Under the Westminster system they can only guess the date of the election, but as soon as the election is called they model the behaviour and timing of the important actors — political leaders, propaganda managers and voters in swing seats.
Respondants to pre-election polls are never asked how they might vote on the day of the election. People change their minds but pollsters have no reliable way for measuring this changeability.
Punters have no reliable way of measuring electors’ changeability. They simple know that it is a critical factor in the outcome of many elections and make an educated guess about it.
In punting, an active market can exist only if there is significant disagreement about the outcome or the price of one competitor is discounted enough to appear to justify a speculative bet.
Punting markets are therefore a negative sum game after commissions, taxes and costs are taken out of the betting pool. On the other hand, in transactions of goods and services both the seller and the buyer can persuade themselves that they have won from the transaction — a positive sum game.
Katz, gamblers do persuade themselves that they are special, that they have better than even odds of prevailing in the long run. It’s just all those other punters that are mugs.